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Abstract — Recently, a negative interplay has been shown to 
arise when scheduling/ AQM techniques and low-priority conges- 
tion control protocols are used together: namely, AQM resets 
the relative level of priority among congestion control protocols. 
This work explores this issue by (i) studying a fluid model that 
describes system dynamics of heterogeneous congestion control 
protocols competing on a bottleneck link governed by AQM and 
(ii) proposing a system level solution able to reinstate priorities 
among protocols. 

I. Problem statement 

It comes to no surprise that our daily activities increas- 
ingly require ubiquitous Internet access. In a typical day, we 
call friends with Skype or Gtalk, socialize on Twitter and 
Facebook, upload pictures to Picasa and Flickr, backup or 
share data with BitTorrent, Dropbox and Mega, and upload 
new tunes to GoogleMusic, etc. Moreover, as a side-effect of 
the proliferation of connected household devices, the need to 
synchronize data between the numerous appliances arises. As 
copies of the data are increasingly stored in some datacen- 
ters, this translates into frequent upload/downloads to/from 
the Cloud. At the same time, the periphery of the Internet 
infrastructure was designed having in mind that users would 
mostly be data consumer (as opposite to data producer), as for 
instance testified by the deployment of Asymmetric Digital 
Subscriber Line (ADSL) in Europe. While this fact was al- 
ready challenged by peer-to-peer traffic (P2P), current uploads 
to the Cloud further clash the infrastructure asymmetry. 

This mismatch may lead to "bufferbloat" [8|, i.e., very large 
queuing delays, up to several seconds, experienced by Internet 
users. While the "persistently full buffer" phenomenon is not 
new ||9l, it has been exacerbated by the ubiquitous presence 
of significantly large buffers at the access (made relatively 
cheap by today's technology), that the loss-based congestion 
control of TCP is apt to fill: as TCP regulates its sending rate 
(halving the sending window) only in occurrence of losses, the 
buffer is forcibly filled upQ. Moreover, while infrastructural 
solutions to the bufferbloat have been proposed in the liter- 
ahire, such as scheduling (SFQ ||26l, DRR ||33) and Active 
Queue Management (RED [14l|, Choke f29l), their adoption 
has been rather limited. The situation has only recently started 
to change, with worldwide operators implementing scheduling 

'Notice that buffer sizes involved are often small in absolute terms (few 
KBs), but are relatively large to capacities of the narrow cable, ADSL or 
WiFi links (few Kbps) in front of these buffers, yielding possibly multi-second 
queueing delays |21|. 



policies in the upstream of the ADSL modem to improve the 
quality of user experience (e.g., in France, Free implements 
SFQ since 2005 [3 1), and with new promising AQM techniques 
under active development such as CoDel |28|. 

At the same time, recent evolution of Internet application 
landscape has also seen a proliferations of new applications 
-or "apps", as the Internet is now often accessed through 
smartphones or portable devices- that control the flow of 
traffic to and from the Cloud. Due to the lack of infrastructural 
solution to the bufferbloat problem, and since deployment of 
all-software solution is much easier with respect to infrastruc- 
tural upgrades, applications have started proposing alternative 
models to the standard TCP best effort congestion control. 
Notable examples include Microsoft Background Intelligent 
Transfer Service (BITS), Picasa background upload option, 
and BitTorrent low extra delay background transport (LED- 
BAT) ||32l . The latter is especially interesting since BitTorrent 
still represents a sizeable amount of Internet traffic and, 
according to Bram Cohen, "LEDBAT is now the bulk of 
all BitTorrent traffic, [...] most consumer ISPs have seen the 
majority of their upload traffic switching to a UDP-based 
protocol" 15|. The rationale behind the design of LEDBAT is 
that user ADSL link represent likely the uplink bottleneck, 
so that congestion is typically self-induced by concurrent 
traffic sessions generated by the same user -such as BitTorrent 
transfers in parallel with Skype call and other Cloud uploads- 
which LEDBAT is designed to avoid. 

Our recent work [18 1 shows, by means of simulation and 
experiments, a negative interplay when scheduling/ AQM tech- 
niques and low-priority congestion control (LPCC) protocols 
such as LEDBAT are combined: namely, AQM resets the 
relative level of priority among congestion control protocols. 
In this work, we first study a fluid model that describes system 
dynamics when flows adhering to heterogeneous congestion 
control protocols, such as LEDBAT and TCP, compete on 
a bottleneck link governed by AQM. Then, we propose a 
system-level solution able to reinstate priorities among pro- 
tocols. 

The remainder of this papers is organized as follows. Sec. |II] 
overviews closest related work. Fluid model is presented in 
Sec. [nil and an extensive set of numerical results, gathered 
on the scenario described in Sec. |IV] are reported in Sec. fVl 
and compared against ns2 simulations. System design of a 
feasible solution is then described in Sec. IVII before Sec. IVIII 
concludes the paper and outlines our next steps. 



II. Background 

It would be extremely cumbersome to comprehensively 
retrace over 20 years of Internet research in these few pages . 
An historical viewpoint is sketched in |17|: we extend this 
viewpoint by reporting in Fig. [T] a timeline of research 
in scheduling/AQM algorithms and LPCC protocols. The 
timeline clearly shows a temporal separation of these two 
research topics, which in our opinion helps understand why 
the AQM vs. LPCC interaction assessed in this paper was 
only barely previously exposed. In this section, we overview 
the related work separately considering (i) the AQM vs. LPCC 
interaction, (ii) fairness of congestion control protocols, (iii) 
LEDBAT and other low priority protocols, (iv) fluid modeling 
of TCP and AQM. 

LPCC: NICE LP LEDBAT 

AQM: SFQ RED DRR Choke O |22] (g) CoDel 
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Fig. 1. Timeline of AQM and LPCC algoritlims. 

A. Interaction 

To the best of our knowledge, aside our previous work [TSl, 
only IIU mentions AQM and a LPCC (namely, LEDBAT) in 
the same paper In one of the tests, the authors experiment with 
a home gateway that implement some (non-specified) AQM 
policy other than DropTail. When LEDBAT and TCP are both 
marked in the same "background class", the "TCP upstream 
traffic achieves a higher throughput than the LEDBAT flows 
but significantly lower than" under DropTail [31]. This is 
known explicitly in flie LEDBAT RFC, stating that under AQM 
is possible that "LEDBAT reverts to standard TCP behavior, 
rather than yield to other TCP flows" [32 1. 

In our previous work JTS], we further show that this 
behavior is general and can arise from the interaction of 
any scheduling/AQM discipline and LPCC protocol shown in 
Fig.H] using a twofold methodology including ns2 simulation 
and experiments from both controlled testbed and wild Inter- 
net. The present work differs from [18] in both its depth and 
methodology: indeed, we adopt a more narrow but profound 
scope, selecting LEDBAT and RED as representative examples 
of the LPCC and scheduling/AQM design space that we then 
analytically model. 

B. Fairness 

Our main focus in this paper concerns fairness of the 
capacity share among heterogeneous control protocols on a 
bottleneck governed by AQM. While fairness is a long studied 
subject, its investigation generally considered rather different 
settings. First, it has often been tackled in the intra-protocol 
case [Q, ifTsll . lfT6l . ll30ll : i.e., heterogeneous settings of a single 
protocol flavor For instance, ifTSl studies RTT unfairness 
of TCP Reno. Similarly, we pointed out the existence of a 



LEDBAT latecomer unfairness issue ll30l - that we show to 
be less relevant in the case of short lived flows and solve for 
backlogged connections in ||7l, lfT6l . 

Fairness in the inter-protocol case, thus closer to ours 
heterogeneous control protocols settings, has long been studied 
as well [HI, lfT2l . [13[. Old works especially focused on 
undesirable side-effect of delay-based congestion control of 
Vegas, that makes it back off in presence of TCP Reno {A\. 
Even more recent work on the topic studies different issues 
than ours. Authors of lfT2l . ifTSi focus on several high- 
speed variants of TCP: in their case, fairness between the 
different protocols is thus desirable, while in our settings 
unfairness would be desirable (as it would imply that low- 
priority property is maintained). Complementary to this work, 
authors in [12[ design and analyze an AQM scheme (named 
AFpFT after Approximate Fairness through Partial Finish 
Tag), that they show via ns2 simulations to reinstate fairness 
in the heterogeneous protocols case [il3j . 

C. Low priority 

Protocols such as NICE [ISl, LP d, 4CP ED and ||20l 
share the same low-priority spirit of LEDBAT. We carried out 
a simulation-based comparison of NICE, LP and LEDBAT 
in ||6l, showing that LEDBAT has the lowest level of priority. 

Some important differences among the above protocols are 
worth stressing. NICE [34] extends the delay -based behavior 
of TCP Vegas with a multiplicative decrease reaction to early 
congestion (detected when the number of packets experiencing 
a large delay in an RTT exceeds a given threshold). Differently 
from LEDBAT, that reacts to instantaneous one-way delay 
(OWD) variations, NICE instead react to RTT variations, thus 
possibly reducing the sending window in one direction due to 
growing delay in the opposite direction. 

LP \22i enhances the loss-based behavior of NewReno 
with an early congestion detection based on the distance of 
the OWD from a weighted moving average calculated on 
all observations. In case of congestion, the protocol halves 
the rate and enters an inference phase, during which, if 
further congestion is detected, the congestion window is set to 
zero and normal NewReno behavior is restarted. This differs 
from LEDBATthat aims at explicitly bounding the maximum 
delay introduced in the bottleneck queue, which is particularly 
important for VoIP, gaming and all other interactive delay- 
sensitive applications. 

D. Fluid modeling 

Other work ||T9], [El], ||25], IET) relate to this as far 
as its methodology is concerned. We point out that, since 
generally a single dominant TCP flavor is modeled ifT9l . ll27l 
(optionally including unresponsive background traffic ll24l or 
short-lived connections ["251), the novelty in this context lies 
in the definition of a fluid model of heterogeneous responsive 
sources, notably including LEDBAT. 

As our main innovation is not on the technique per se, but on 
its application to the study of a particular problem, we resort 
to classic models for TCP [24] and RED [27[, that we extend 
to incorporate novel popular protocols such as LEDBAT. 
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Fig. 2. Network scenario 



III. Fluid model 



We describe the network scenario we model with the help of 
Fig. |2] A user device generates a number of long-lived flows 
competing on the same bottleneck of capacity C, with a buffer 
of size B full size packets. Applications on the device either 
use a best-effort TCP congestion control, or a lower-than-best- 
effort LEDBAT contiol. We denote the TCP and LEDBAT 
window at time t as W{t) and Z{t) respectively. 

For TCP, we neglect the slow-start phase, which is instead 
only optional in LEDBAT. As such, we limitedly model the 
TCP congestion window behavior in congestion avoidance 
phase. At the reception of the (n + l)-th ack at time tn+i, 
the TCP congestion window is updated as follows: 
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if packet loss, 
otherwise 



(1) 



As for LEDBAT, it reacts to losses by halving the congestion 
window as TCP does, but otherwise its congestion window 
increase is not larger than TCP ramp-up in congestion avoid- 
ance, and more precisely is proportional to the distance of the 
queuing delay q{t) from the delay target r: 
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if packet loss, 
otherwise 



with the current queuing delay q{t) measured as: 

Q_{^'n) D{tyi^ Djyi^n 
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where D{tn) represents the instantaneous one-way delay 
(OWD) estimate, while the base delay Z3„ii„ is the minimum 
observed OWD. The rationale is that, over a sufficiently 
large number of observations D,nin accurately represents the 
fixed component of the delay (i.e., propagation delay plus 
negligible transmission delay, which should be the one found 
when queues are empty) so that the D{tn) — Dmin difference 
represents the variable component of the delay (i.e., queuing 
delay plus negligible processing delay). 

Notice that host synchronization over the Internet is known 
to be hard. As such, it is worth stressing that the OWD estimate 
D{t„) is affected by an unknown clock offset between the 
two endpoints, and is thus of no practical use. Conversely, the 
offset cancels in the difference operation in (|3]l, which is only 
affected by clock drift - that is of much smaller magnitude 
and furthermore easier to correct [ITTI . 



Furthermore, whenever the queuing delay hits the target r, 
the congestion window settles, i.e., lim^^T- = 0. 

A. Ordinary Differential Equation System 

To analyze the interactions between sources and queue 
dynamics, we adopt a continuous time fluid approach |19|, 
ll24l . II25I . IIZTI in which the average dynamics of both 
sources and queues are described by deterministic Ordinary 
Differential Equations (ODE). To write the ODE system, we 
denote by Wi{t) the instantaneous congestion window at time 
t for connection i in the fluid system, by Ri (t) the Round Trip 
Time (RTT) and by p{t) the packet dropping probability at the 
buffer We consider the case of Nw TCP and Nz LEDBAT 
connections sharing the same bottleneck, where flow-level 
congestion window evolution the TCP case is adopted from 
GJI: 
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In accordance with RED specifications the packet dropping 
probability at the buffer, p{t), is a function /(•) of the 
estimated average queue size Q{t), with: 



(2) fiQ) = 




Q < mmth 

minth <Q< maxt/i 

Q > muxth 



Q{t) is obtained by q{t) through an exponential weighted 
moving average from samples taken every S seconds. The fluid 
equation that relates Q{t) to q{t) is given as in 1271 by: 
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dt S ' ' S 

Notice that in general case, the RTT R^{t) = Tp^i + q{t)/C 
cannot be considered to be constant since the component due 
to queuing delay can be predominant over the propagation 
delay q{t)/C > Tp^i - which is especially true in case of 
bufferbloat due to FIFO buffering. Conversely, in case AQM 
is used, it could be reasonable to assume the reverse q{t) /C < 
Tp,i to hold. 

B. Equilibrium point 

The equilibrium point of the above dynamical sys- 
tem is given by the stationary (i.e. constant) solution 
iW* ,Z* ,q* ,Q*) of the system of differential equations, (|5j, 
(|6]l, O and ([8]l. In our case, we are going to prove the existence 
of at most one unique equilibrium point. We remark that the 
existence of the equilibrium point can be always granted by 



properly setting the RED parameters. For the sake of simpUcity 
we consider a homogeneous case, i.e. a case in which all 
the connections exhibit the same RTT, however we would 
like to remark that the extension to a more general case is 
straightforward. 

From ^ and ^ respectively, with simple algebra we 
obtain: 
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where we have ignored the upper/lower clipping effects on the 
window size in both LEBDAT and TCR From O we obtain 
that q* (with q* G [0,maxt/i]), has to be the solution of the 
equation: 



fj — ^ if 9 < T 

if 9* > T 



(12) 



Observe that the existence of at most one solution for (fTzt 
is granted by the fact that while the expression on the left is 
weakly increasing with q (it takes the value for q — 0), the 
expression on the right is, instead, weakly decreasing (being 
strictly positive for any q > 0). Thus, a unique solution exists 
iff: 
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if r > maxt,i (^3-) 
if T < maxf/i 



Observe that by properly setting maxj, we can always meet 
(fTsT l. indeed the term on the left tends to infinite as maxp — 1. 

As long as RED is configured to keep the queue shorter 
with respect to the LEDBAT target (i.e., as long as q* < r) 
a non-perfect prioritization between TCP flows and LEBDAT 
flows is experienced, indeed LEBDAT flows are still able to 
grab a non-negligible fraction of the bottleneck bandwidth. As 
discussed in Sec. IIVI this is the most likely case in practice. 

IV. Scenarios 

A. User applications 

We argue that the most challenging scenario, in terms of 
matching results gathered via simulation and fluid mode, is 
the one with few number of flows. This is intuitive since 
in the case of multiple backlogged connections, statistical 
multiplexing will smooth out the impact of events, such as 
TCP retransmission time out, that would otherwise cause 
discontinuities in the case of few connections. At the same 
time, we also argue that the most practically relevant scenario 
is precisely one with a relatively small number of flows. 
Indeed, since the bottleneck sits at the user access, the number 



of concurrent connections will be bound, even considering 
multiple applications/users in the household. 

We consider both the Cloud and the P2P cases. In the Cloud 
case, it is easy to see that a small number of connections will 
be opened, at any given time, for a specific service. While 
considering a single user, even the server contacted will evolve 
over time (e.g., due to load balancing), this likely happens 
over time-scales that are much larger with respect to the short 
time-frames that we consider as "backlogged" data transfers 
(i.e., from tens of seconds to minutes) in this paper Hence, 
the number of backlogged connections is upper-bounded by 
the number of Cloud services the user subscribes to, such 
as DropBox for data, GoogleMusic for music and Picasa 
for pictures/videos. Additionally, the number of simultaneous 
connections also depends on the on/off synchronization pattern 
toward the Cloud. As users are not continuously generating all 
kind of data at the same time, it thus reasonable to envision 
only a moderate number of concurrent backlogged connections 
per household, some of which may be lower priorities (e.g., 
pictures) over others (e.g., critical data, backup). 

Consider next the P2P case, where it makes sense to 
consider file-sharing applications such as BitTorrent due to 
its popularity, and since it introduced LEDBAT in the first 
place precisely due to the bufferbloat problem. In BitTorrent, 
pipelining of piece requests at the apphcation-level can cause 
multiple chunks to be transmitted consecutively over the 
same connection at transport-level. Since BitTorrent limits the 
number of concurrent slots to abouH 4 per torrent, the number 
of concurrent connections will be again small. Moreover, Bit- 
Torrent peers periodically evaluate the throughput toward other 
peers every 10s of seconds, and connections are maintained 
in case of good end-to-end throughput: coupled to pipelining, 
this entails that over the tens of seconds to minute timescale, 
connections can be considered backlogged. 

From the above discussion, in the following we will limit- 
edly consider an equal number N — Nw — Nz of flows, 
and let the total number of flows vary in 2N e [2, 10] 
range. Unless otherwise stated, we consider homogeneous 
RTT delay settings with propagation delay Tp = 50 ms (to 
which we add a jittering component of 1 ms to avoid synchro- 
nization of the congestion window dynamics). To precisely 
characterize system equilibrium properties, we will let the 
LEDBAT target r vary - that in the uTorrent implementa- 
tion of LEDBAT, this can be easily done by tweaking the 
net . utp_target_delay settings. 

B. Network configuration 

Without loss of generality, we consider a single access 
bottleneck and fix the capacity to C= 1Mbps, typical range for 
ADSL/Cable access. The bottleneck buffer can accommodate 
up to B = 100 packets that, considering P=1250 Bytes sized 
packets for simplicity, corresponds to a maximum queuing 
delay of 1000 ms. Notice that these values are commonplace 
nowadays, with modem buffers able to hold up to 4 seconds 

^The limit actually increases with the square root of the uplink capacity 
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Fig. 3. Reprioritization phenomenon: Time evolution of W{t), Z{t) and Q{t) under DropTail (a) and RED (b,c,d) for different values of r. 



worth of traffic [21]. To precisely characterize system equi- 
Ubrium properties, we explore variations of the RED settings 
minth , maxt/i (in packets), maxp settings to cover the full 
supporj|. For convenience, we may express the target r in 
milliseconds or packets: notice that due to our settings, a 
packet is worth 10 ms of queuing delay. 

As previously observed, the reprioritization phenomenon 
vanishes in case q* > t: in other words, when the queue 
size at the equilibrium exceeds the LEDBAT queuing delay 
target t, all LEDBAT flows will by design yield to TCP and 
the system will behave as 1191 . fT7\ . At the same time, this 
scenario is unlikely to hold in practice. Consider indeed that 
end-to-end congestion control protocols such as LEDBAT rely 
on noisy measures of queuing delay, so that they will not be 
able to guarantee protocol efficiency when r — !• 0. 

Then, notice that for the typical ADSL transmission speed 
of 500Kbps, the transmission of a full MTU packet takes about 
24 ms: initial versions of LEDBAT used to set r = 25 ms, i.e., a 
packet worth of queuing. However, due to practical limitations 
(including timestamp precision in Windows OS, clock drift of 
several ppm in off-the-shelf PCs, etc.) this setting did not allow 
to fully exploit the link capacity, reason why the target was 
later increased to t = 100 ms. While a 100 ms target may be 
reasonable for an end-to-end protocol, an AQM may be more 
precise in measuring the queue size and in adopting more 
aggressive dropping policy (e.g., minimi < maxt;^ < r for 
RED, or lower packet sojourn time than r for CoDel), so that 
it is reasonable to assume that the target AQM queue size will 
he q* < T in practice. 

V. Results 

In this section, we present and discuss numerical results of 
the ODE describing the system dynamics. Numerical results 

'Notice tliat we do not aim at providing tuning guidelines of RED, wliicli 
is notoriously difficult 1 10| and scenario dependent [JJJ], but rather to provide 
thorough characterization of the equilibrium. 



are gathered either (i) finding roots to the equilibrium equation 
via bisection method or (ii) integrating the ODE via Runge- 
Kutta, and are organized as follow. After a description of the 
scenario (Sec. llVl i. we depict the temporal system evolution to 
show the reprioritization phenomenon (Sec. IV-A) that we will 
investigate further at the equilibrium (Sec. IV-B) . We then carry 
out a sensitivity analysis (Sec. IV-CI l and discuss local conver- 
gence properties (Sec. IV-DI l of the model. Finally, we validate 
a subset of the numerical results against those obtained from 
ns2 simulator (Sec. IV-Eb using our own implementation of 
LEDBAT, which is available as open source at (1T|. Validation 
is performed on the most challenging (in terms of matching 
the simulation vs. fluid model results) and relevant scenarios 
(in terms of practical relevance). 

A. Reprioritization 

We start by showing the time-evolution of the system 
equations when Nw = Nz = 1 in Fig. |3]under either DropTail 
(a) or RED (b,c,d) discipHnes. Top plot shows the W{t), Z{t) 
and W{t) + Z{t) congestion windows evolution, while queue 
Q{t) is reported in bottom plots. 

In the DropTail case, we set t ==10 packets and observe 
the same behavior shown via ns2 simulation in ll30l : i.e., 
LEDBAT yields to TCP as expected under DropTail. In the 
RED case, we set maxf/j = B = 100, mint/; = 10,maxp = 1 
for the sake of illustration and let r grow from 10 (b) to 20 (c) 
and 50 (d) packets. Notice that in case (b), RED drastically 
reduces the queue size and let TCP window fluctuates at about 
the capacity. Yet, when the target increases in (c) and (d), 
LEDBAT becomes increasingly aggressive under RED, and 
competes more fairly against TCP. 

To avoid cluttering the pictures, we instead avoid reporting 
the behavior of LEDBAT for increasing target r under Drop- 
Tail: from the sensitivity simulation-based sensitivity analysis 
reported in Q, it emerges that LEDBAT yields to TCP for a 
large range of t < B values, and only whenever r approaches 
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Fig. 4. Equilibrium analysis of TCP share ratio p* at the equilibrium as 
a function of r/minth for various flow number N, LEDBAT r, and RED 
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(or exceeds) the buffer size B LEDBAT behavior becomes 
loss-based as TCP. 

Shortly, in the following we will refer to this difference in 
LEDBAT aggressiveness with respect to TCP as a "repriori- 
tization" phenomenon induced by RED, which indeed resets 
the relative level of priorities between LEDBAT and TCP. 

B. Equilibrium 

While it is hard to get closed form solution of the equi- 
librium point, we can numerically find roots of the ODE 
equations via the bisection method. We now characterize 
the reprioritization as a function of system parameters. For 
convenience, we define the TCP share ratio as the ratio 
between W{t) and Z{t): 

W{t) 



Pit) 

at the equilibrium we have: 



W{t) + Z{t) 



(14) 



(15) 



Notice that in Fig. [3] we have purposely selected settings 
that show that the system may actually fluctuate around the 
equilibrium point (maxp — 1), though for many settings 
the equilibrium is actually smoothly reached. We depict in 
Fig. m the TCP share ratio p* at the equilibrium for varying 
user scenarios (i.e., number of TCP and LEDBAT flows N , 
LEDBAT target settings t, and RED settings minj/j and 
maxp). 

Fig. |4] shows that under AQM the TCP share exhibits a 
sharp transition phase as soon as t exceeds wmth, quickly 
dropping with an hyperbolic slope from a monopoly situation 
(p* — 7> 1 for values of r close to q*) to a fair share {p* « 0.58 
for T = 2g*). Interestingly, fSl shows that in the DropTail 
case, a sharp transition phase from TCP monopoly to a fair 
share happens whenever t ^ B. This difference is rooted on 
the fact that RED dropping rates are strictly positive as soon 
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Fig. 5. Sensitivity analysis of TCP share ratio p* and Queue size q* at the 
equihbrium as a function of r for various flow number A'^, LEDBAT t, and 
RED mini(,,maxp settings. 



as the queue size exceeds miiit/i, whereas DropTail decisions 
have to wait until the queue exceeds B. 

C. Sensitivity 

From Fig.|4]we also gather that different RED settings yield 
only minimally affect the reprioritization phenomenon. For 
completeness, we depict in Fig. |5] the values of the queue 
q* (top plot) and the TCP share p* (bottom plot) at the 
equilibrium for several RED settings. This time, q* and p* are 
reported directly as a function of r. (i.e., we avoid normalizing 
over the RED vcvmth parameter) Trivially, since no dropping 
happens for q < rainth, this parameter plays the biggest role in 
determining the queue size at the equilibrium. Next comes the 
load factor, i.e., number of flows insisting on the bottleneck, 
followed by the maximum dropping probability maxp of the 
RED profile. 

The impact of the LEDBAT target r on the queue size 
has almost a step-like behavior, that can be explained taking 
into account that LEDBAT flows activate only when t > q* 
(or, T > uiinth given the above remark). Recalling the 
sharp transition phase in LEDBAT aggressiveness as soon as 
T > miiit/i, the impact of LEDBAT flows after activation is 
to increase the load profile, about as a TCP flow would do. 

Bottom plot of Fig.|5]reports similar information to previous 



Fig. m although lines are now clearly separated for different 
rainth profiles. We stress that large values of minth > 50 
could avoid the reprioritization, but at the price of an already 
sizeable bufferbloat. 

D. Convergence 

We now observe evolution of the primitive W{t), Z{t), q{t) 
variables and of the p{t) observable toward the equilibrium 
W* , Z* ,q* , p* . Some examples of trajectories are shown in 
Fig. |2] In more details, top plots report the case for initial 
conditions 1^(0) = Z{0) = g(0) = 0, while bottom plots 
report the trajectories of 100 random initial conditions. For 
convenience, we express the relative error with respect to the 
equilibrium, so that we can superpose multiple trajectories on 
the same graph. 

Top-left plot shows the relative distance of {W{t),p{t)) 
from the equilibrium (W*,q*), while top-right plot shows 
{W{t),q{t)), for two different scenarios. Especially, it can 
be seen that after an initial oscillation, the queue converges 
to the equilibrium (also reflected in the breakdown), while 
convergence is smoother for the other variables. 

Bottom-left plot considers 100 random initial conditions and 
focuses on the initial phase (t < t') of the system evolution 
shown in the top-left counterpart. Similarly, the bottom-right 
plot considers 100 random initial conditions but focuses on 
later times when the system is about to reach convergence 
(t > t"). 

While further steps are necessary to prove the local system 
stability (e.g., studying a linearized version of the system at the 
equilibrium, which is part of our ongoing work), this simple 
visual inspection has already provided useful insights about 
the convergence of the equilibrium point for different initial 
conditions and scenarios. 

E. Validation 

We confirm the validity of the model by contrasting in 
Fig. |6] the value p* of the TCP share at the equilibrium 
against simulation results obtained via our own LEDBAT 
ns2 implementation [1]. We point out that we have already 
extensively analyzed the reprioritization phenomena via both 
experiments and simulations [IS], making the ns2 scripts 
available at ID to reproduce the phenomenon. Hence, our 
main aim here is not to provide a coverage of those results, 
but rather to validate the most representative instance of our 
results - which is clearly represented by the TCP share ratio 
that precisely quantifies the reprioritization. 

As we have previously seen, minth has by far the biggest 
role in determining the TCP share curve, followed by the 
number of flows in the bottleneck and by maxp at last. At 
the same time, while the traffic scenario depends on the user 
and is a free parameter, from the discussion in Sec. |lV]we do 
not consider minth as a free parameter, while maxp is less 
interesting to study due to its more limited impact. 

Hence, we fix miiifh, = 10, maxth = B ~ 100, maxp = 0.1 
and consider two traffic scenarios Nw — Nz = {1; 5}. Fig. |6] 
contrasts average simulation results (solid point, with standard 
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Fig. 7. Convergence to the equilibrium. 



deviation bars over multiple runs) against the equilibrium (fT2l i 
previously discussed (dotted fine) and a slightly more accurate 
version (solid black line) that compensates two simplifications 
of the fluid model that we discuss next. Notice indeed that ( fT2l l 
captures reasonably well the essence of the reprioritization 
phenomenon. Still, two quantitative discrepancies arise. 

First, it can be seen that for values of t > minth the model 
underestimates the TCP share. This results from a known 
problem of the TCP model presented in f24l that this work 
extends: i.e., ll24j is known to underestimate TCP congestion 
window with respect to simulation, which can be easily 
compensated by taking into account a multiplicative decrease 
factor of 1.5 (instead of 2) as in 1,24 1 . The refined equilibrium 
takes into account this correction, and is significantly more 
accurate when r > minth- 

Second, recall that when t < q*, the model degenerates 
into a simpler one in which only TCP flows compete on the 
bottleneck, hence p* = 1. In practice however, we know that 
LEDBAT will keep sending a minimum of 1 packet per RTT: 
this is done to continuously measure the queuing delay, at very 
low frequency and intrusiveness. LEDBAT does this in order 
to promptly react to queuing delay reduction and effectively 
utilize the spare capacity as soon as the link becomes free 
again. Hence, in case t < q*, a refined estimation could 
upper bound p* by reducing the capacity available for TCP 
proportionally to the number of LEDBAT flows, i.e.. 



p*<l- 



N 



CTp 
P 



The refined equilibrium takes into account also this second 
correction, and is significantly more accurate with respect to 
(fT2] i when r < minth- Yet, we argue that such low level of 
detail can be better captured with ns2 simulations, and that 
quantifying the exact level of reprioritization is less relevant 
for practical purposes - i.e., as users will likely be interested in 
knowing whether their non-critical bulky transfers are indeed 
lower-priority with respect to critical continuous backups, or 
if they compete on a roughly equal basis. 
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VI. System-level solution 

Recent evolutions on the Internet applications and infras- 
tructure seem to suggest that AQM and Low priority conges- 
tion control (LPCC) protocols will have to coexist: indeed, 
popular applications are developing delay-based congestion 
control protocols such as BitTorrent/LEDBAT on the one hand, 
operators are starting to deploy AQM/scheduling on the user 
access uplink on the other hand. As such, it is imperative to 
find solutions to the negative AQM/LPCC interplay we have 
shown in this paper While a general solution is hard to find, 
as testified by the current standpoint after over 20 years of 
research, a patch to this specific problem may be within reach. 

Some might argue that small buffers would be enough to 
solve bufferbloat altogether Yet, there are several reasons why 
this simple solution is not sufficient. First, in presence of 
too small buffers, it would be difficult for TCP and other 
congestion control to fully saturate the capacity, causing an 
undesirable efficiency loss. Second, deciding a buffer size 
is a matter of concern per se: consider indeed WiFi links, 
where the capacity may fluctuates widely over time, so that 
no single buffer size can at the same time (i) be large enough to 
support TCP congestion control and (ii) rule out bufferbloat 
in a fast-to-slow transition from 54Mbps to 2Mbps. Finally, 
jeopardization of relative priorities are not solved by small 
buffers |18|. 

An ideal solution should achieve two goals: (i) meet quality 
of service constraints while (ii) respecting relative levels 
of priorities among protocols. Quality of service constraints 
clearly translate into upper-bounding the queuing delay, that 
we know is used by protocols to enforce their relative pri- 
orities. Since even a single TCP flow may bufferbloat the 
others, the solution needs AQM, as otherwise the quality of 
service constraints would be violated. At the same time, to 
avoid the LPCC reprioritization phenomenon, we argue that 
classification capabilities will be needed in AQM to account 



for flows' explicitly advertised level of priority. 

Although in the more general case classification has failed 
to be adopted (IP TOS field, DiffServ, etc.), and the ability 
to claim higher priority could be easily gamed, in a hybrid 
AQM vs. LPCC world it makes sense for flows to claim a 
lower priority. We believe that this subtle difference can make 
an important practical difference in terms of deplorability. 

A simple way would be to let application exploit IP TOS 
field. While the overloading of the IP TOS can be troublesome 
within an operator network, this is not an issue in the home. 
Indeed, the usefulness of the IP TOS is not end-to-end but 
merely meant as a low-priority signal to the box in the user 
home. Hence, IP TOS could be leveraged by the ISP CPE in 
the user home to apply differential treatment to best-effort and 
low-priority traffic (e.g., different AQM loss profiles, different 
scheduling weights), after which the end-user IP TOS value is 
no longer useful and can be rewritten by the CPE (or at the 
DSLAM, or BRAS, etc.) in the network of an operator using 
DiffServ if needed. 

We further stress that the firmware governing home-routers 
and WiFi APs is generally based on some variants of the Linux 
kernel, possibly open-source as in the OpenWrt or CeroWrt 
cases. We point out that the above solution is therefore already 
implementable without any additional development effort - 
e.g., using strict priority queuing or shaping. In the Linux 
traffic control (tc) suite, this can be achieved with the PRIO 
queuing discipline (qdisc) that implements non-shaping con- 
tainer for a configurable number of classes which are dequeued 
in order This first solution allows for easy prioritization of 
traffic, where lower classes are only able to send if higher ones 
have no packets available. A second solution offered by Linux 
tc is represented by the CBQ qdisc that offers shaping and 
finer-grained prioritization capabilities. 



VII. Conclusions and future work 

This work models an interdependency phenomenon between 
heterogeneous congestion control protocols and active queue 
management. Specifically, in case a low-priority congestion 
control protocol (e.g., LEDBAT) competes against TCP on a 
bottleneck link governed by AQM (e.g., RED), the relative 
level of priority of the congestion control protocols is reset, 
and the protocols compete on a roughly equal basis. 

We model the problem as a system of Ordinary Differential 
Equations that we solve numerically and validate against 
simulation results. Our main results is that the TCP share at 
the equilibrium equals p* = (l + ■\/(t — (7*)/t) , where t 
is LEDBAT target queuing delay and q* is the length of the 
queue at the equilibrium determined by RED settings. 

By reason of increasing deployment of both low-priority 
congestion control and AQM techniques, the problem may 
be of significant practical relevance. As we believe that it 
may be desirable for end-users (or end-user applications) to 
autonomously and coarsely set their relative level of priorities, 
we have proposed simple yet effective system-level design and 
practices that can solve the issue we have characterized in this 
paper. 

Our future work moves along two main paths. On the one 
hand, we aim at refining our investigation by means of a 
control theoretic analysis of the properties of the linearized 
system, to e.g., prove local stability of the equihbrium. On 
the other hand, as we know by |18 | the problem to hold in 
general for several AQM and LPCCs, another line of work 
goes in the direction of extending the model in both the AQM 
(e.g., CoDel {2^) and low-priority congestion control (e.g., 
NICE t34J) directions. 
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